DOCUMENT RESUME 



ED 453 038 



RC 022 985 



PUB TYPE 
EDRS PRICE 
DESCRIPTORS 



AUTHOR 



TITLE 



PUB DATE 
NOTE 



IDENTIFIERS 



Stenhoff , Donald M. ; Menlo ve, Ronda; Davey, Bryan; 
Alexander, Melina 

Preference of Students 1 Response and Outcomes of Distance 
Education Course Evaluation. 

2001-03-00 

8p . ; In: Growing Partnerships for Rural Special Education. 
Conference Proceedings (San Diego, CA, March 29-31, 2001); 
see RC 022 965. 

Reports - Evaluative (142) -- Speeches/Meeting Papers (150) 

MF01/PC01 Plus Postage. 

Course Evaluation; ^Distance Education; Higher Education; 
♦Special Education; Special Education Teachers; Student 
Surveys; *Teacher Education; *Teacher Effectiveness; 
♦Teacher Student Relationship; Teaching Experience; 
Teleconferencing; World Wide Web 

♦Technology Utilization; *Web Based Instruction; Web Sites 



ABSTRACT 



A Web-based survey examined student rankings of 



teacher-student and student- technology interactions in distance education 
courses. Respondents were 61 students taking undergraduate or graduate 
distance education courses in special education at a Carnegie Research I 
university. Five instructors taught these courses. Three were full-time 
faculty members who had used an interactive teleconferencing system before, 
but two were new to the Internet-delivered interactive teleconferencing 
system used for these courses. Two instructors who co-taught a course were 
first-time instructors in a higher education setting, were adjunct faculty, 
and were new to distance education. Concerning how effectively instructors 
used the technology delivery system, the instructor with the most experience 
received the highest student ratings, followed by the two other instructors 
with prior distance education experience. The two instructors with the least 
amount of distance education experience received the lowest ratings. This 
pattern was repeated on other questions concerning teacher-student 
interactions. A majority of students found the course Web site useful, but 
again the instructor with the most distance education experience received the 
highest ratings. Most students strongly agreed that the course Web site was 
easy to use. However, the class having the students with the least experience 
in distance education received lower rankings on this question. (TD) 
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PREFERENCE OF STUDENTS’ RESPONSE AND OUTCOMES OF 

DISTANCE EDUCATION COURSE EVALUATION 
oo 
co 

° Distance education is growing rapidly especially in higher education settings. More and more students are 

enrolling in classes that are delivered over a technologically-based system. In fact, a report by The International 
I? Data Cooperation (IDC) estimated that 2.2 million college students will be enrolled in distance education by 2002, 

§ an increase from approximately 710,000 in 1998 (Council for Higher Education Accreditation, 1999). 

Student learning in distance education courses is comparable with traditional student learning in on-campus 
courses (Souder, 1993). While student learning is comparable, more research is needed in the area of instructor 
effectiveness in delivering distance education courses and student satisfaction with distance education courses. We 
feel that it is too simplistic to examine distance education as a whole. There are several instructional components 
that are integrated in distance education. The purpose of this survey was to gain students’ rankings of teacher- 
student and student-technology interactions. 

Methods 



Participants & Setting 

The participants were students at a Carnegie Research I university enrolled in at least one of four 
undergraduate or graduate distance education courses in special education (N=103). All of the students surveyed 
were enrolled in a Bachelor’s of Science, Master’s, or Doctoral program in Special Education at the university. 
Students’ ages ranged from 22-46+ and ranged in class ranking from juniors to doctoral levels. About 85% of the 
respondents were female and about 1 5% were male. All of the students had taken at least one distance education 
course, and the majority had taken as many as six or more (Table 1). 

Table 1 

Student Demographics 



Course Number & Percentage of 
Responses 







SPED 


SPED 


SPED 


SPED 






5060 


5070 


5340 


6700 






N=12 


N=13 


N=13 


N=23 




Gender: 

Male 


9% 


8% 


15% 


35% 




Female 

Age: 


91% 


92% 


85% 


65% 




18-21 


0% 


0% 


0% 


0% 




22-25 


9% 


8% 


8% 


10% 




26-29 


18% 


8% 


8% 


10% 


kn 


30-33 


18% 


23% 


23% 


25% 


GO 


34-37 


0% 


8% 


8% 


15% 


€T> 


38-41 


36% 


46% 


46% 


5% 


42-45 


9% 


8% 


8% 


10% 


CNJ 


46+ 


9% 


0% 


0% 


25% 
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BEST 



best copy available 



Class Ranking: 



Freshman 


0% 


0% 


0% 


0% 


Sophomore 


0% 


0% 


0% 


0% 


Junior 


0% 


8% 


0% 


0% 


Senior 


100% 


91% 


80% 


6% 


Master’s 


0% 


0% 


20% 


72% 


Doctorate 


0% 


0% 


0% 


22% 


Number of distance 










education courses taken: 










1 


0% 


0% 


0% 


21% 


2-3 


9% 


0% 


0% 


26% 


4-5 


9% 


8% 


8% 


21% 


6 or more 


82% 


92% 


92% 


32% 



Five instructors taught these courses. Three instructors were full-time faculty at the university and had 
taught using an interactive audio video teleconferencing system before. However, two of these instructors were new 
to the internet-delivered audio video teleconferencing system used while teaching the following courses, “Teaching 
Math to Students with Mild/Moderate Disabilities (SPED 5340), and “Single-Subject Research Design” (SPED 
6700). The third instructor, who taught “Policies & Procedures in Special Education” (SPED 5070), has taught 
several courses using the internet-delivered audio video teleconferencing system. Two instructors co-taught 
“Consulting with Parents & Teachers” (SPED 5060). They were first time instructors in a higher education setting, 
were adjunct faculty, and were new to distance education (Table 2). Three courses were at the undergraduate level; 
one was a graduate level course. 



Table 2 

Course Number, Title, Enrollment, and Instructor Experience 



Course 

Number Course Title Enrollment Instructor Experience 



SPED Consulting with Parents & Teachers 24 • 1 st class taught at university level 

5060 • No experience with distance 

education 
Co-taught class 
Adjunct faculty 

Taught several classes using 
internet-delivered audio video 
teleconferencing system 
Faculty 

Taught prior distance education 
course 

1 st time using internet-delivered 
audio video teleconferencing 
system 
Faculty 



SPED Single-Subject Research Design 


33 


Taught prior distance education 


6700 




course 




• 


1 st time on internet-delivered audio 






video teleconferencing system 




• 


Faculty 



SPED Teaching Math to Students with 24 

5340 Mild/Moderate Disabilities 



SPED Policies & Procedures in Special 22 

5070 Education 
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Classes were transmitted to approximately seven sites across the state. Distances ranged from on-site to 250 
miles. The on-site room had two monitors in the room. One monitor displayed the instructor, and the other displayed 
the graphics, such as the PowerPoint slides the instructor used for lectures. Students on-site could also see the 
instructor directly. Instructors sat in the front of the room and could see the on-site students physically, i.e., without 
the aid of a monitor. 

Off-site rooms had a conference table, multiple tables, or desks for students to sit at. The computer monitor 
was placed in a location where all students could view it. Microphones were placed in a location that students could 
access them for site-to-site communication. 

Procedures 

The study questionnaire was conducted using a web-based survey instrument that was available two ways. 
Students could access the survey by clicking on a hyperlink in the course website that would take the student 
directly to the survey instrument, or via an email hyperlink sent directly to the students. When students clicked on 
the hyperlink, it took them directly to the survey instrument. 

In each of the surveyed classes, a researcher spoke to the students explaining that they would be receiving a 
survey in an email message and could also access the survey through the course website. Additionally, the 
researchers expressed the importance of completing the survey to improve the quality of distance education courses 
at the university. The survey was initiated 1 5 days prior to the end of the semester. Students had access to the survey 
for a total of 1 5 days. 

Nonrespondents were emailed a second time 5 days before the survey was closed as recommended by Gall, 
Borg, and Gall, 1996. The email again included the hyperlink to the survey instrument. 

Apparatus 

Delivery system . Classes were delivered using a synchronous, internet-delivered two-way audio video 
teleconferencing system (Sorenson EnVision). Using this system, the instructor was able to see the students at all of 
the sites simultaneously via monitors stationed in front of the instructor. Students were able to see the instructor and 
graphics, e.g., PowerPoint slides, simultaneously during instruction. On-campus students could view the instructor 
on one monitor and graphics on another monitor. Students at remote sites viewed the instructor and graphics on one 
computer monitor. The monitor displayed a “talking head” in the left comer frame of the monitor screen and the 
graphics in a larger frame in the middle of the monitor screen (Figure 1). Instructors had the ability to change the 
talking head frame so the students could see students at other sites. Some instructors may have used this option more 
frequently than others. In order for students to talk, they had to retrieve the microphone, press a button on the 
microphone, then talk. The microphone then sent their audio to all of the other sites. 
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Figure I. Graphic representation of computer monitor screen viewed by students at remote sites. 



Survey instrument . Survey questions were compiled from previous course evaluation surveys and included 
additional items the authors identified as related to measuring instructor and student interactions. A 4-point Likert- 
type format was chosen for the survey (Gall, a al., 1996). The benefit of using this type of survey format is that it 
allows the students to specify the degree to which they hold a particular view (Hayes, 1998). Survey questions may 
be found in Tables 3 and 4. 

Web-based survey service . The survey was conducted using a web-based service called Zoomerang that 
may be found at www.zoomerang.com . The service provides tools to gather and analyze feedback. After sending a 
survey instrument, the results are held in a private and secure environment. Users are able to customize or re-write 
templates, and launch the surveys with personalized greetings (Zoomerang, 2001). The cost of the basic service is 
free. 

Data Collection 

Data were collected from the survey website and percentages of responses were tabulated. Overall data for 
each course survey were printed from the Zoomerang website. Printouts were made for each student’s survey 
responses and results were then compiled by class. 

Results 

Teacher-Student Interactivity 

One hundred and three surveys were distributed through email. There was a total response rate of 59% 
(N=61). Ninety-three percent (N=57) of the surveys were completed via email hyperlinks. Only four students (7%) 
accessed the survey via the hyperlinks located on the course websites. Results are listed in Tables 3 and 4. 

Table 3 

Student Responses to Questions Regarding Interactivity by Percent 



Percentage of Responses 





Questions 




SPED 

5060 

N-12 


SPED 

5070 

N=13 


SPED 

5340 

N=13 


SPED 

6700 

N=23 


1 . 


The instructor effectively used the 


Strongly Agree 


9% 


62% 


31% 


16% 




technology delivery system. 


Agree 


36% 


38% 


62% 


63% 






Disagree 


45% 


0% 


8% 


11% 






Strongly Disagree 


9% 


0% 


0% 


11% 


2. 


Students were given opportunities to 


Strongly Agree 


27% 


85% 


46% 


74% 




ask questions. 


Agree 


55% 


15% 


54% 


26% 






Disagree 


18% 


0% 


0% 


0% 






Strongly Disagree 


0% 


0% 


0% 


0% 


3. 


Students were provided opportunities 


Strongly Agree 


18% 


69% 


38% 


19% 




to express opinions, make comments 


Agree 


64% 


31% 


46% 


21% 




and share ideas. 


Disagree 


18% 


0% 


15% 


0% 






Strongly Disagree 


0% 


0% 


0% 


0% 


4. 


Class time was used efficiently. 


Strongly Agree 


9% 


62% 


23% 


11% 






Agree 


18% 


23% 


54% 


42% 






Disagree 


55% 


15% 


23% 


32% 






Strongly Disagree 


18% 


0% 


0% 


16% 


5. 


Questions asked via telephone, i.e., 


Strongly Agree 


10% 


69% 


23% 


11% 




voicemail, were answered in a timely 


Agree 


50% 


31% 


69% 


74% 




manner. 


Disagree 


20% 


0% 


8% 


16% 






Strongly Disagree 


20% 


0% 


0% 


0% 
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6. Questions asked via email were 


Strongly Agree 


9% 


85% 


38% 


22% 


answered in a timely manner. 


Agree 


45% 


15% 


54% 


61% 




Disagree 


18% 


0% 


8% 


11% 




Strongly Disagree 


27% 


0% 


0% 


6% 



Eighty-five percent of the students who responded in SPED 5070 recorded a “Strongly Agree” on Question 
1, “instructor effectively used the technology delivery system,” (Table 3). This was followed by SPED 5340 and 
SPED 6700 students with 31% and 16% respectively. However, 62% and 63% of students in SPED 5340 and SPED 
6700, respectively, reported that they “Agree” with Question 1 . Over half the students in SPED 5060 reported they 
at least “Disagree” (45%) and “Strongly Disagree” (9%) with Question 1 (Table 3). 

When asked if they were “given opportunities to ask questions” (Question 2) and to “provide opportunities 
to express opinions, make comments, and share ideas” (Question 3) student ratings were highest for the experienced 
instructor (SPED 5070) with 85% and 69%, respectively. Student ratings were also high for SPED 6700 with 74% 
and 79%, respectively. The majority of students across all the classes recorded levels of at least “agree” for these 
two questions. However, 18% of students enrolled in SPED 5060 recorded “Disagree” when asked Question 2. This 
was the only class in this question who recorded a “Disagree” with the question (Table 3). 

Question 4 asked if “class time was used efficiently.” There was a significant difference in students’ ratings 
of this question. Student ratings for SPED 5070 was 62% (Strongly Agree). The second highest was 23% for the 
SPED 5340 course; however, student ratings for “Agree” was 54%. Ratings by students for SPED 5060 were 55% 
“Disagree” for Question 4 (Table 3). 

Student ratings of Questions 5 and 6, asking whether questions were answered in a timely manner via 
voicemail and email also were highest for the experienced instructor (SPED 5070) with 69% and 85%, respectively. 
The two other faculty members’ students recorded 69% (SPED 5340) and 74% (SPED 6700) for Question 5. 
Students’ ratings for SPED 5340 and SPED 6700 faculty members for Question 6 recorded a 54% and 61% for 
“Agree”, respectively (Table 3). 

Student-Technology Interactivity 

Sixty-nine percent of the students who responded in 5070 recorded a “Strongly Agree” on Question 7, 
regarding usefulness of the website for the course (Table 4). This was followed by the SPED 6700, SPED 5340, and 
SPED 5060 students with 47%, 31%, and 9% respectively. Additionally, students’ ratings for “Agree” on Question 7 
were 64% for SPED 5060 and 46% for SPED 5340 courses. See Table 4. 

The majority of students accessed the website weekly as seen on Question 8, “frequency I accessed the 
website.” Students’ ratings were 64%, 54%, 69%, and 75% for “Agree” across the courses (Table 4). 

Question 9 asked, “I found the website easy to use. Students’ ratings in courses SPED 6700, SPED 5060, 
SPED 5070, and SPED 5340 were 63%, 64%, 77%, and 77%, respectively. Three “Strongly Agree” for Question 9 
(Table 4). 



Table 4 

Student Responses to Questions Regarding Interactivity with Course Technology 



Percentage of Responses 



Questions 




SPED 

5060 

N=12 


SPED 

5070 

N=13 


SPED 

5340 

N=13 


SPED 

6700 

N=23 


I found the website useful for my 


Strongly Agree 


9% 


69% 


31% 


47% 


course. 


Agree 


64% 


31% 


46% 


26% 




Disagree 


18% 


0% 


23% 


21% 




Strongly Disagree 


9% 


0% 


0% 


5% 
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8. The frequency I accessed the website 


Daily 


27% 


38% 


31% 


15% 


was: 


Weekly 


64% 


54% 


69% 


75% 




Monthly 


9% 


8% 


0% 


10% 


9- I found the website easy to use. 


Strongly Agree 


64% 


77% 


77% 


26% 




Agree 


36% 


15% 


23% 


63% 




Disagree 


0% 


8% 


0% 


11% 




Strongly Disagree 


0% 


0% 


0% 


0% 



Discussion 

This survey process gathered rankings of instructor and student interactions from students enrolled in 
distance university special education classes at a public university. Survey instruments were emailed to each student 
or were accessed via the course website. This survey has several limitations. First, the measure was not validated 
prior to initiation (Gall, et al., 1996). Second, it is possible that not all students were able to access the survey 
instrument on the Internet. Students who answered the survey may differ from those who did not. However, the 
survey does provide the authors with insights into the current state of distance education in the Special Education 
Department as well as direction for future study. 

Weston and Amundsen (2000), discussed several issues that need to be considered in on-line instruction. 
Three of these considerations include: student access, interactivity, and navigation. The survey instrument addresses 
each of these considerations. Questions 7, 8, and 9 addressed student access to the course website, student- 
technology interactivity, and navigation (Table 4). Questions 1 through 6 addressed teacher-student interaction 
(Table 3). 

Ninety-three percent of the students accessed the survey using the hyperlink that was directly emailed to 
them. This factor may have implications in the method instructors use to interact with their students outside of class 
time. Another factor that may have contributed to this outcome was the convenience of having the hyperlink sent 
directly to the student instead of the student searching for the hyperlink on the course website. 

Overall, teacher-student interaction ratings appear to be higher in the course where the most experienced 
instructor taught (SPED 5070). It also appeared to be lowest in the course where the least experienced instructors 
taught (SPED 5060). This may be due to a number of factors. First, it may be the amount of experience of teaching 
at the higher education level. Secondly, it may be the amount of experience teaching using the distance education 
medium. 



Question 1 asked how effectively the instructor used the technology delivery system. As seen in Table 3, 
the instructor with the most experience received higher student ratings than the other instructors. The two instructors 
who had prior distance education experience (SPED 5340 and SPED 6700) but not on the internet-delivered audio 
video teleconferencing system received the second highest ratings. Furthermore, we see that the instructor with the 
least amount of distance education experience (SPED 5060) receiving the lowest ratings. Prior experience with 
distance education might be the result of the ratings. 

The questions relating to “opportunities for students to ask questions” and “providing opportunities to 
express opinions, make comments and share ideas”, Questions 2 and 3, resulted in higher ratings for the experienced 
teacher (SPED 5070) and higher ratings for the SPED 6700 teacher (Tables 2 and 3). A student rating worth noting 
is found in Table 3, Question 2. The teachers who had the least amount of teaching experience (SPED 5060) had a 
rating of 18% “Disagree” while no other teachers received this rating for their courses. This may also be related to 
the factor of experience instructing and with the distance education medium. Microphone access may have 
contributed to student ratings as well. Some students may be apprehensive to ask for the microphone, or possibly 
speaking over the internet-delivered audio video teleconferencing system. This provides need for future research 
regarding these possible factors. 

Efficient use of class time student ratings also provided some interesting data. The students’ ratings 
indicated that the three faculty members used class time efficiently. However, the SPED 5060 course with the first 
time teachers instructing, had student ratings of 55% of the students recording “Disagree” on this question (Table 3). 
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Questions 5 and 6, which students rated for “voicemail and emails answered in a timely manner” again had 
student ratings that favored the experienced teacher (SPED 5070). A possible reason from this may include 
techniques that allowed the instructor to respond more efficiently to student questions or the instructor may have, 
over time, developed a system for responding to voice and email. 

A majority of the students’ ratings indicated that they found the course website useful for their course. But 
here again the instructor with the most distance education experience (SPED 5070) received the highest percentage 
of students strongly agreeing with Question 7 (See Table 4). Possible explanations for this difference in students’ 
ratings may be attributed to instructor understanding of the course website. Other possible explanations may include 
prompting from the instructor to use the website for various information and assignments that required website 
interaction to complete. 

Question 9 asked for students’ ratings of the ease of use of the course website. Overwhelmingly, the 
majority of students in the SPED 5060, SPED 5070, and SPED 5340 courses rated that they “Strongly Agree” that 
the course website was easy to use. In the SPED 6700 course, however, the majority of students only “Agree” with 
the ease of use. Examination of Table 1 shows that the students in SPED 6700 reported having far less distance 
education courses than the other three. Again, experience becomes a possible issue. 

In light of the current study possible questions for future study surface. Does instructor effectiveness in the 
use of the technological system have a relationship to higher rates of interactivity between students and teachers? Do 
the lower ratings for the first time instructors indicate a need for training in distance education prior to instructing? 
What is the relationship between technology training and higher student ratings? What are effective strategies for 
increasing teacher-student interaction both during class and for questions answered via voice and email? There is 
need for replication of this survey in different disciplines, instructors and delivery systems. Additionally, it is 
necessary to expand the numbers of survey respondents to examine trends in student rankings. 
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